03. Methods for Feature Extraction

Heading

Methods for feature extraction

ND320 C3 L3 03 Methods For Feature Extraction

Video Summary

We have discussed some methods for feature extraction such as 2D, 2.5D, and 3D convolutions.

A convolution is an operation that applies a convolutional filter to each pixel of an image. A more detailed explanation could be found in this great Wikipedia article.

A simple 2D convolution with a 3x3 kernel could be visualized by the following animation:

Convolution

Animation of image convolution with a 3x3 convolutional filter [[1,0,1],[0,1,0],[1,0,1]]

Animation of image convolution with a 3x3 convolutional filter [[1,0,1],[0,1,0],[1,0,1]]

Summary

We have just discussed several types of convolutions that could be used for feature extraction of 3D medical images:

2D Convolution is an operation visualized in the image above, where a convolutional filter is applied to a single 2D image. Applying a 2D convolution approach to a 3D medical image would mean applying it to every single slice of the image. A neural network can be constructed to either process slices one at a time, or to stack such convolutions into a stack of 2D feature maps. Such an approach is fastest of all and uses least memory, but fails to use any information about the topology of the image in the 3rd dimension.

2.5D Convolution is an approach where 2D convolutions are applied independently to areas around each voxel (either in neighboring planes or in orthogonal planes) and their results are summed up to form a 2D feature map. Such an approach leverages some 3-dimensional information.

3D Convolution is an approach where the convolutional kernel is 3 dimensional and thus combines information from all 3 dimensions into the feature map. This approach leverages the 3-dimensional nature of the image, but uses the most memory and compute resources.

Understanding these is essential to being able to put together efficient deep neural networks where convolutions together with downsampling are used to extract higher-order semantic features from the image.

Next up, we will take a closer look at how convolutions operate by running through a notebook and performing an exercise.

How many bytes do you need for 2d conv?

Assuming that you keep input and output images in memory, what is the minimum amount of memory in bytes that you need to allocate in order to compute a convolutional feature map of size 28x28 from an image of size 30x30 using a 3x3 convolutional kernel? Assume 16 bits per pixel and 16 bits for each parameter of the kernel.

SOLUTION: 3400